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Abstract 

Online citizen science offers a low-cost way to strengthen the infrastructure for scientific research and engage members of 
the public in science. As the sustainability of online citizen science projects depends on volunteers who contribute their 
skills, time, and energy, the objective of this study is to investigate effects of motivational factors on the quantity and quality 
of citizen scientists' contribution. Building on the social movement participation model, findings from a longitudinal 
empirical study in three different citizen science projects reveal that quantity of contribution is determined by collective 
motives, norm-oriented motives, reputation, and intrinsic motives. Contribution quality, on the other hand, is positively 
affected only by collective motives and reputation. We discuss implications for research on the motivation for participation 
in technology-mediated social participation and for the practice of citizen science. 
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Introduction 

Recent years have seen a substantial growth in the scale and 
scope of technology-mediated social participation (TMSP) projects 
[1], such as Wikipedia, Linux and CiteULike, which rely on 
volunteers who contribute their time, energy and skills for the 
creation of a public good [2] . The present study focuses on one 
type of TMSP: technology-mediated (or 'online') citizen science. 
Like other TMSP efforts, citizen science projects enable members 
of the public to take part in scientific research addressing real- 
world problems [3], often through web-based contribution [4], 
Various aspects of scientific research - mainly data collection and 
analysis - are labor-intensive, time-consuming, and consequently 
costly. Online citizen science reduces the costs of scientific 
research, increases the resources available to research teams, 
fosters a partnership between citizens and scientists, and enhances 
public understanding of science. Scientific breakthroughs such as 
the discovery of a pulsar by Einstein@home volunteers [5] and the 
achievements made by contributors to Foldit, an online game in 
which fold proteins into chemically stable configurations [6], 
illustrate the potential of this participatory approach to science. 
Citizen science projects can be characterized by the different levels 
of task granularity: "the smallest possible individual investment 
necessary to participate in a project" required from contributors 
[7]. Task granularities range from the minimal investment 
required from participants in the case of volunteer computing 
projects, to more active and demanding tasks in projects that 
involve distributed data gathering and analysis. 

In distributed analysis projects, volunteers engage in image 
classification or analysis in a variety of scientific areas, which often 
require task-specific training. Examples include projects like 



Stardust@home (http://stardustathome.ssl.berkeley.edu) in which 
volunteers analyze images of interstellar dust particles, and Galaxy 
Zoo (http://www.galaxyzoo.org), in which volunteers classify 
images of galaxies [8] . This type of contribution requires some 
training, and ongoing cognitive effort in analyzing images. A 
second category of citizen science - distributed data gathering 
projects - involve geographically distributed inputs from a large 
number of sources. One example is the Citizen Weather Observer 
Program (CWOP) in which volunteers provide real-time weather 
data that is used by research institutions and the weather services. 
This type of contribution requires offline effort of buying and 
setting up a weather station, as well as ongoing maintenance of the 
equipment. Finally, volunteer computing represents the third 
category, which is based on dividing a large computational task 
into small tasks that are then distributed over the Internet and 
completed on volunteers' computers. Well known volunteer 
computing projects include SETI@home (http://setiathome.ssl. 
berkeley.edu) and Folding@home (http://folding.stanford.edu). 
This type of contribution requires the download and installation of 
a software application, after which contribution is largely passive, 
and therefore the task granularity is relatively low. 

Online citizen science is based on two pillars: (1) a technological 
pillar, which involves developing computer systems to manage 
large amounts of distributed resources, and (2) a motivational 
pillar, which involves attracting and retaining volunteers who 
would contribute their skills, time, and effort to a scientific cause. 
While the technological dimension has been widely studied, the 
motivational dimension of citizen science received little attention 
to date. 

In similar settings, the motivations for contributing to TMSP 
projects have been studied in variety of projects [9,10], including 
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Wikipedia [11-13], Amazon consumer reviews [14], and open 
source software (OSS) development [15-17]. Nonetheless, it is not 
clear whether findings from other TMSP settings would carry over 
to online citizen science settings, as these exhibit some distinctive 
characteristics that are likely to affect the contributors' motivation. 
First, in online citizen science there is a clear distinction between 
the volunteers making the contribution and those benefiting from 
the aggregate effort (i.e. the scientists who run the project). This 
asymmetric structure differs from most other TMSP projects (e.g. 
Wikipedia, YouTube), where the distinction is blurred. Second, 
the lengthy duration from the time volunteers make their 
contribution to the time scientific output is made public differs 
substantially from TMSP projects, where the contributions are 
viewable immediately, and these differences are likely to influence 
contributors' motivation. Third, a single contribution to an online 
citizen science project is sometimes too small to be attributed to a 
specific individual, whereas in other communities the deliverables 
(e.g. text, code, or photos) can stand on their own and are usually 
attributable to their contributor. These differences further stress 
the need to investigate motivations for participation in the specific 
context of citizen science [18]. Moreover, results from recent 
studies of motivation for participation in TMSP demonstrate that 
the salience of the various motives differ across projects. For 
example, extrinsic motives (e.g. career advancement, reputation) 
have been shown to be an important determinant of participation 
in the case of photo tagging [19], but had no significant effect on 
the time contributors spent in OSS development [15], and these 
extrinsic motives had a significant negative effect on engagement 
in Wikipedia [11]. Thus, it is not clear the extent to which findings 
from other TPSP projects would carry over to online citizen 
science. 

The objective of this study is, thus, to investigate the 
motivational drivers of participation in online citizen science 
projects. We focus our attention on two outcome variables: (i) the 
quantity of one's contributions (i.e. the number of contributions 
made in a given period) and (ii) the quality of contributions. With 
the scant research on TMSP and contribution quality, we expect 
the comparison of the motivational factors affecting contributions' 
quantity vs. quantity to yield insights applicable beyond the 
domain of citizen science. 

The paper continues as follows: the next section reviews related 
work on volunteers' motivations for participation in TMSP, and 
particularly in citizen science settings; the section that follows 
develops hypotheses about the relationships between motivations 
on one hand and contribution quality and quantity on the other; 
we then describe the research methodology and continue to 
present the results of our empirical evaluation; the next section 
discusses our findings, and highlights implications for theory and 
practice; the final section concludes the paper, pointing to future 
research directions. 

Background 

Contributions' Quantity and Quality. The success of 
TMSP projects depends not only on the amount of users' 
contribution, but also on its quality [20]. Quality, in this context 
is "a degree of excellence of what is produced", whereas quantity 
"refers to the total amount of what is produced" ([21] p. 53). 
While both quantity and quality important, research on the factors 
underlying TMSP has primarily focused on the quantity of 
contribution, whereas the drivers of quality have been under- 
explored, perhaps because the subjective nature of quality and the 
difficulty to measure it. 

Prior applied psychology research has revealed the tradeoff 
between quantity and quality as a fundamental characteristic of 



human performance in cognitive tasks [21-23]. Although both of 
these dimensions of human performance are often desirable, due 
to the limited cognitive resources and information processing 
capability of an individual they are not always in harmony, as 
observed in the context of information processing tasks [24] and 
goal directed behaviors [22], as well as in the context of TMSP 
(examples include prior studies of discussion forums [20] and 
online product reviews [14]). 

Our study focuses on motivation as an antecedent of both the 
quantity and quality of contribution to citizen science projects. 
Empirical evidence suggests that motivational factors may exert 
differential effects on quantity and quality of contribution in online 
communities. For example, a study of professional networks of 
practice found that in the context of an online discussion forum, 
expectations of reciprocity had a significant effect only on 
contribution quantity, whereas altruism influenced the quality of 
contributions [25]. Other studies found that social affiliation, 
utilitarian motives, and self-expression only affected the quantity of 
online product reviews submitted, while altruism, reciprocity, and 
skill development had a significant effect on quality [14]. A related 
construct - commitment to the community - was shown to have a 
positive effect on the quantity of photos posted to a photo sharing 
community, while self-development motives affected the quality of 
contribution [26] . It is therefore expected that motivational factors 
exert differential effects on quality and quantity in online citizen 
science projects. 

Motivational Factors Affecting Contributions' 
Quantity. Sustained participation and the sharing of individu- 
als' knowledge are critical for the viability of all online 
communities [27,28], and thus an understanding of contributors' 
motivations for participation [29] is essential for successfully 
designing and managing TMSP efforts [30,31]. In recent years, a 
growing number of studies have investigated volunteers' incentives 
for sharing information across a wide range of online communities, 
such as open source projects, Flickr, Twitter, and Wikipedia [1]. 
Some of the important factors that were found to affect 
participation include the improvement of skills and enhancement 
of status [16,32], enjoyment [19], reciprocity [25], identification 
with contributors' community [11,15], and group level factors 
such as social network properties [33] and group membership size 
[27]. 

Within the context of online citizen science, research on 
motivations for participation is still in an early phase and empirical 
evidence is scarce. Holohan and Garg [34] studied the motivations 
for participating in volunteer computing projects and contributing 
one's computer resources to SETI@home. They found the most 
salient motivational factors to be the desire to help scientific 
research, reputation, and gaining technical knowledge, while social 
factors (being part of team and maintaining social ties) had only 
secondary importance. A study of motivation for participation in 
the analysis of galaxy images performed at the Galaxy Zoo project 
identified ten motivational categories, including: excitement, 
learning, desire to discover, social interaction, use the project as 
a resource for [8] teaching, the beauty of the images, fun, 
amazement by vast scale of the universe, desire to help, interest in 
the project, interest in astronomy, and interest in science in 
general. Both these studies provide a descriptive analysis of 
contributors' motivation, but no attempt is made to link 
motivations to behavior. A study of SETI@home volunteers 
[35] on the other hand, developed a research model of the factors 
determining volunteer computing users' contribution. The results 
of testing the model exposed personal enhancement and team 
affiliations to be positively related to computing resource 
contribution. Overall, however, prior studies have focused on (a) 
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single setting contribution, and (b) contribution quantity rather 
than quality. What seems to be missing is a broader study of 
different citizen science settings, focusing on both quantity and 
quality of contribution. 

Motivational Factors Affecting Contributions' 
Quality. A study among authors of Amazon consumer reviews 
[14] found altruism and reciprocity to be positively related to 
reviews' quality, whereas a study of Flickr users [26] found the 
motivation of learning to be positively related to contribution 
quality. In a more work-related setting, a study of the motivational 
factors associated with postings in a professional network of 
practice [25] found that reputation and enjoyment in helping 
others were positively related with the helpfulness of the postings 
as perceived by their recipients. 

Theoretical Context 

Motivation for social participation. Research on partici- 
pation in social movements has a long tradition in the social 
sciences [29,36,37]. Social movements can be defined from a 
psychological point of view, as "effort [s] by a large number of 
people to solve collectively a problem that they feel they have in 
common" ([38], p. 5). Traditional social movements and 
technology-mediated social participation are all based on funda- 
mental principle of voluntary contributions made as part of a 
collective effort, explain why theories of social movement 
participation have been used in the study of TMSP [11,15]. 

A theoretical framework that integrates central findings from 
social movement research has been developed by Klandermans 
[39,40]. According to this framework, the motivation or willing- 
ness to participate in a social movement is viewed as a function of 
the expected costs and benefits from participation. Since the goals 
of such movements often benefit members of the public regardless 
of whether they participated in the collective action, the 
achievement of the goal may be insufficient as a motivating force 
in and of itself (resulting in social loafing or lurking in online 
communities). Individuals are therefore likely to have other 
reasons for participating, which can involve both social and 
material costs and benefits. Klandermans describes three motives 
for participation in social movement, with each of the three 
reflecting a different type of expected cost or benefit: (a) collective 
motives associated with the importance one attributes to the 
collective goals of the movement; (b) social motives resulting from 
expectations regarding the reactions of important others - such as 
friends, family or colleagues - to one's participation; and (c) reward 
motives linked to potential benefits to be gained from participation, 
such as gaining reputation, or making new friends. All three 
motives are assumed to contribute positively to the willingness to 
participate in collective action organized by the social movement. 

Our study extends the social movement participation model by 
drawing on Self-Determination Theory [41], in line with recent 
studies of motivation within TMSP [11]. At the most fundamental 
level, SDT contrasts extrinsic motivation (in which individuals 
engage a task in order to achieve a desired outcome) with intrinsic 
motivation (in which individuals engage in a task out of interest or 
enjoyment). While Klandermans' discussion of reward motives 
emphasizes extrinsic rewards [39,40], intrinsic motives represent 
an alternative type of reward and were found to play an important 
role in TMSP [1 1,15,42]. Thus, we include both types of reward 
motives in our study of citizen science. In order to provide a clear 
focus, we use in our study one specific type of extrinsic factor, 
reputation, as this factor has been shown to be an important driver 
of participation within TMSP in a variety of projects [25,43,44]. 
For clarity reasons, and in line with [15], we refer to the motives 



related to the expected reactions of important others such as family 
and friends as norm-oriented motives. 

In sum, the framework used in our study includes these four 
motivational factors — collective motives, norm-oriented motives, intrinsic 
rewards, and reputation - as antecedents of both the quantity and 
quality of participation in online citizen science projects. 

Hypotheses Development. Collective motives have been 
found to be important factors in social movements [39] and 
volunteering [45] . For instance, Simon and colleagues [46] report 
on two studies of social movements where collective motive had a 
significant effect on the willingness to participate. In the context of 
TMSP, however, the effects of collective motives have been less 
consistent. In Wikipedia, Nov [12] reported statistically insignif- 
icant correlation between Wikipedia ideology and participation, 
and similarly, Schroer and Hertel [1 1] found the effect of 
collective motives on engagement to be insignificant. In the 
context of OSS development, on the other hand, collective motives 
seem to be more salient. Lakhani and Wolf [16] found that a third 
of OSS developers studied considered collective motives impor- 
tant. Hertel and colleagues [15] reported that collective motives 
had a significant effect of on "the willingness to be involved in the 
future" in OSS contribution. In citizen science, similarly to OSS 
development, participation is organized around projects, each 
having a distinct goal, and a collective identity that is derived from 
the project's goal and ideology. For example, participants in the 
Stardust@home project refer to themselves as "dusters". Such 
high identification with the project's ideology is often a key 
determining factor in volunteers' decision to join a project, as well 
as a driver of ongoing participation. In line with the effects 
observed in open source projects [15,16], we therefore expect that 
collective motives in citizen science projects will have positive 
impact on the quantity of participants' contributions. Formally 
stated: 

Hla: Higher level of collective motive will be associated with greater 
contribution quantity. 

Collective motives can also influence the quality of contribution. 
While the notion of 'quality of contribution' is not directiy 
applicable to social movements (for which the Klandermans model 
was originally intended), and the antecedents of contribution 
quality in TMSP have also been underexplored, there is some 
evidence suggesting that when contributors share the projects' 
values their contribution to it will be of a higher quality. For 
example, in OSS development, values associated with the project, 
such as sharing information and helping others, were found to be 
related to trust and quality of communication among contributors, 
which were in turn related to project success [47]. We, thus, expect 
collective motives to affect quality of contributions in citizen 
science projects, and we hypothesize: 

Hlb: Higher level of collective motive will not be associated with a higher 
contribution quality. 

Social norms play an important part in enforcing participation 
in support communities [48] and social movements [39,49], In 
addition, the Information Systems literature shows a positive 
relation between subjective norms (i.e. the perceived social 
pressure to engage or not to engage in a behavior) and intended 
usage behavior [49] . However, empirical evidence of the effects of 
social norms on volunteering is mixed. For example, Piliavin et al. 
[50] found that individuals donate blood due to external, social 
motives, while Simon et al. [46] found in both his studies of 
participation in social movements that norm-oriented motives had 
only a marginally significant effect on the 'willingness to 
participate'; finally, Houle et al. [51] found that the salience of 
norm-oriented motives differs between volunteering tasks. Such 
mixed evidence is also apparent in studies of TMSP. For example, 
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norm-oriented motives did not have a significant effect of the 
willingness to be involved in the future in OSS projects [15], nor 
did they have a significant effect on Wikipedia engagement [11]. 

We believe that the mixed evidence of the effect of norm- 
oriented motives could be attributed - at least to some extent - to 
an under-specification of the outcome variable. Perceived social 
pressure from important others, such as friends and family 
members, could encourage one to register and contribute to a 
project from which everyone can benefit, but are less likely induce 
the kind of commitment, enthusiasm and sustained effort that are 
necessary for making high-quality contribution. In other words, 
individuals whose participation is driven primarily by perceived 
social pressure may not be doing their best as far as contribution 
quality goes. Such a basic level of participation may be sufficient 
for an individual to communicate to important others that he is 
now also a member of their volunteer project. We, thus, expect 
that norm-oriented motives may affect the quantify of contributions 
made to online citizen science projects, but not the qualify. 
Formally stated: 

H2a: Higher level of norm-oriented motives will be associated with a higher 
contribution quantity. 

H2b: Higher level of norm-oriented motives will not be associated with a 
higher quality of contribution. 

Social exchange theory [52] posits that individuals engage in 
social interaction based on an expectation that it will lead in some 
way to social rewards such as approval, status, and respect. 
Specifically, reputation is an important asset that an individual can 
leverage to achieve and maintain status within a collective [53] 
and one way in which an individual can benefit from active 
participation in group activity is through the enhancements of his 
personal reputation. In corporate settings, results from prior 
research on electronic networks of practice provide evidence that 
building reputation is a strong motivator for active participation 
[54]. In the context of social movements and voluntarism, 
perceived individual costs and benefits associated with the 
voluntary engagement are an important driver of participation 
[39]. For example, Gidron [55] found that young volunteers 
tended to view their volunteer work as a self-development 
experience, and Beale [56] suggested that students are interested 
in volunteering as a stepping stone to employment. 

Prior TMSP research demonstrates the enhancement of 
personal status in the community is associated with the amount 
of participation in online communities [19,57], as well as in OSS 
development [16,42]. In online citizen science projects, volunteers' 
level of participation is often measured and presented to the 
contributor and to other volunteers. These measures commonly 
take into consideration both the quality and quantity of 
contribution and are a useful tool for enticing participation. For 
example, Starust@home displays on the project website a list of 
the top 100 project volunteers by their score - the number of 
images classified correctly minus the number of images classified 
incorrectly, and BOINC provides details of the quantity of 
computing resources contributed by volunteers over time. We 
therefore anticipate that: 

H3a: Higher level of reputation will be associated with a higher quantity of 
contribution. 

H3b: Higher level of reputation will be associated with a higher quality of 
contribution. 

Enjoyment has been established as one of the prominent factors 
explaining volunteering and charitable behavior [58] and is part of 
the personal benefits to participation in social movements [39] . In 
the context of online communities, enjoying the act of sharing has 
been shown to be a prominent reason for contributing to OSS 



projects ([15,16,42,59], Wikipedia [11,12] and Amazon online 
consumer reviews [14]. We, therefore propose: 

Hypothesis 4a: Higher level of intrinsic motivation will be associated with a 
higher quantity of contribution. 

While the enjoyment associated with contribution is likely to 
drive volunteers to increase their volume of activity, it is not clear 
that enjoyment would enhance the quality of output in the context 
of online citizen science. The tasks performed by volunteers at 
TMSP projects are often mundane tasks in which quality work 
requires the investment of additional effort, paying special 
attention to detail, and satisfying certain requirements of the task 
that may not be enjoyable. As a consequence, intrinsic motives are 
unlikely to be associated with enhanced quality. For example, the 
main task at Stardust@home involves watching images, searching 
for signs of very small interstellar dust particles. Because of the 
nature of volunteers' tasks in online citizen science, we expect that 
intrinsic motivation would not lead to enhanced quality of 
contribution, and we hypothesize: 

Hypothesis 4b: Higher level of intrinsic motivation will not be associated 
with a higher quality of contribution. 

Intrinsic and extrinsic motivations are not additive as standard 
economics assumes; rather, there is a complex relationship 
between the two, often involving crowding effects [60-63]. 
Crowding effects can be subdivided into a crowding out and a 
crowding in effect. The crowding out effect involves a negative 
relationship between intrinsic and extrinsic motivation, and is 
typical of settings where external incentives are perceived to exert 
control over members of an organization, such that self- 
determination is reduced and intrinsic motivation is undermined. 
The crowding in effect, on the other hand, involves a positive 
relationship between intrinsic and extrinsic motivation. An outside 
intervention through rewards or feedback strengthens intrinsic 
motivation in a trusting environment when members' relationship 
with the organization are reciprocal and their goals are aligned, 
such that the intervention is perceived to support intrinsic 
motivation. Prior studies provide substantial empirical evidence 
for both types of crowding effects [64,65]. 

Crowding-in effects has been reported in the study of how 
constitutional and legal rules affect citizens [64]. Intrinsic 
motivation and civic virtue are bolstered when public laws convey 
the notion that citizens are trusted, such that citizens hold 
extensive rights, determine their own participation and can 
organize to influence decision-making processes. Crowding effects 
have not been studied in the context of citizen science projects. 
Nonetheless, given that citizen science projects normally assume 
that volunteer contributors are trusted and that these volunteers 
are free to determine their participation levels, we can expect a 
crowding in effect, such that extrinsic motivation (and specifically, 
reputation motives) would bolster intrinsic motivation. Formally 
stated: 

Hypothesis 5: Higher level of reputation motivation will be associated with 
increased level of intrinsic motivation. 

Methods 

Data collection 

We collected data from three different citizen science projects 
spanning different task granularity levels. Our primary study was 
conducted at Stardust@home, where we investigated the motiva- 
tional factors driving both the quantity and quality of contribu- 
tions. Stardust@home (http://stardustathome.ssl.berkeley.edu/) is 
a citizen science project based at U.C. Berkeley Space Sciences 
Lab, in which over 25,000 participants classify images from 
NASA's Stardust spacecraft, searching for tracks left by very small 
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interstellar dust particles impacting Stardust's aerogel tiles. To 
enable the identification of the particles, the tiles were scanned, 
and the scans were divided to small high-resolution images, which 
were then uploaded to the Stardust@home web-based citizen 
science system. A "virtual microscope" was developed by the 
Stardust@home team (see Figure SI) as an interface through 
which citizen scientists can change the image resolution and point 
at the location of the suspected tracks. Citizen scientists who join 
the project undergo a short online training session and then are 
required to pass a test. After passing their test, the citizen scientists 
are given online access to the Stardust@home "virtual micro- 
scope" and performed the image analysis task. Citizen scientists 
are awarded positive and negative points for correct and incorrect 
classifications of tracks and these points are available publicly on 
the site's leader board. 

In addition, we collected data on the antecedents of the quantity 
of contributions at two other citizen science projects: 

The Citizen Weather Observer Program (CWOP) is a weather 
monitoring citizen science project operated by the National 
Oceanic and Atmospheric Administration. Volunteers to this 
project, who installed low-cost weather stations in their homes, 
record and share weather data online. The data contributed are 
then made available to various organizations including the US 
Weather Service, NASA, and a number of research institutions. 

The Berkeley Open Infrastructure for Network Computing 
(BOINC) is a U.C. Berkeley-based platform running volunteer 
computing projects in various scientific fields. It matches 
volunteers who contribute computing resources to scientific 
projects in need of computational power. To participate, 
volunteers install an application that is then used for managing 
their computer's allocated tasks. After the initial set-up, contribu- 
tion is done automatically, with no interaction with the system. 
Volunteers can set (and change) their level of contribution in a 
number of ways, for example, by determining the amount of disc 
space, memory and CPU time allocated to the citizen science 
project. The BOINC system computes the amount of computer 
power donated by a participant and allocates computation credit 
to participants accordingly. BOINC's unit of credit, the Cobble- 
stone is 1/200 day of CPU time on a reference computer that does 
1,000 MFLOPS (eventually, credit may reflect network transfer 
and disk storage as well as computation). Please see http:/ /boinc. 
berkeley.edu/wiki/Computation_credit for details. Contributors' 
credit is used as a reputation mechanism: the credit is displayed on 
both the contributor's computer and the BOINC's web site, 
motivating participants to contribute more computer resources 
and consequently build their reputation in the community. 

Identical versions of the web-based survey, which differed only 
in the projects' names, were administered to contributors in all 
three citizen science projects (see Appendix B).While our focus 
with respect to sampling was to reach to active project participants 
differences in the sizes of the target populations, as well as access to 
volunteers' email addresses required the use of different sampling 
strategies: A random sample of 4954 BOINC volunteers who were 
active in the three months prior to the survey launch, were sent an 
invitation to participate in the study. 513 Stardust@home who 
were active in the three months prior to the survey launch were 
sent a similar invitation. At CWOP, where email addresses were 
not available to the project leadership, an invitation to participate 
was posted on the project forum by the project leader. Overall, 
139 Stardust@home volunteers, 2390 CWOP volunteers and 
1843 BOINC volunteers participated in the survey, representing 
response rates of 27.1%, 22.1% and 37.2% respectively, much in 
line with the response rates in prior studies in this area [66,67]. 



After cleaning of data we were left with 1202 BOINC and 1837 
CWOP and 139 Stardust@home valid responses. 

Measurement 

A survey was developed based on the Klandermans Model [39] 
and additional sources [1 1,15,42,57,59], relying primarily on prior 
studies that have adapted the Klandermans Model to TMSP 
settings [11,15]. The measurement of collective motives was 
adapted from [15]. Norm-oriented motives were measured based 
on [15] The measure for reputation was adapted from [15,42,57] 
Intrinsic motives were measured using the scales from [1 1,15,59]. 
Contributions' quantity was operationalized as the intention to 
increase participation, in line with the information systems 
research tradition of using behavioral intention as a proxy for 
behavior [68]. The survey items were adjusted to the citizen 
science context, in consultation with projects' leadership. In 
addition, the survey included questions about the age, gender, and 
computer expertise of participants, and these data were used as 
control in our analysis. Participants were asked to rate the 
importance of the different motives (collective, norm-oriented, 
reputation, and intrinsic) on a 1-7 Likert scale. A pilot survey was 
conducted first, and some items were slighdy reworded based on 
feedback received from participants. The quality of contribution at 
Stardust@home was operationalized by using the project's 
internal measure of Sensitivity, a measure of how well volunteers 
correcdy identify tracks. Sensitivity is defined as the number of 
tracks a volunteer correcdy identified, divided by the total number 
of images movies they have searched in which there were tracks. 
Of the 139 participants who were surveyed, 69 were active 
throughout the twelve months that followed the survey, and their 
log data was used in the analysis of contribution quality. As 
common in studies of online communities in which participation is 
highly skewed [31,69,70], we log-transformed the sensitivity data 
for the analysis. 

Ethics Statement 

The study was approved by the Institutional Review Board of 
the Polytechnic Institute of New York University. 

Results 

Descriptive statistics 

Descriptive statistics are presented in Table 1. On average, 
across all three projects, collective motives were rated highest (6.26 
out of 7), followed by intrinsic motives (5.88). Norm-oriented and 
reputation motives were found to be of secondary importance 
(4.56 and 3.64 respectively). 

Measurement Model: Convergent and Discriminant 
Validity 

The convergent validity of our measurement model was 
assessed in several ways, and repeated for each of the three 
projects (at Stardust@home, CWOP, and BOINC). First, we 
examined several competing measurement models to see if they 
provided a better explanation for our data. This is particularly 
relevant for the set of self-reported measures, where the possibility 
of common-method variance introduces an alternate measurement 
model [71]. Confirmatory factor analyses were then performed 
using LISREL 8.80 [72]. In total, we compared three different 
measurement models: (a) the null model (all indicator variables are 
independent), (b) a one-factor model (all indicator variables load 
on a single factor), and (c) a six-factor model measurement model: 
four indicators of motivational constructs, one indicator of 
contributions' quantity, and one indicator of the control variable 
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Table 1. Descriptive statistics for all three projects. 









Stardust @home 


CWOP 




BOINC 




Construct 


Item 


Range 




STD 


Mean 


STD 


Mean 


STD 


Collective Motives 


Coll 


[1..7] 


6.43 


0.87 


6.09 


0.93 


6.21 


1.09 




Col2 


[1-7] 


6.45 


0.76 


6.13 


0.85 


6.30 


1.02 


Norm-Oriented Motives 


Novl 


[1..7] 


4.78 


1.34 


5.10 


1.20 


4.20 


1.41 




Nov2 


[1-7] 


4.87 


1.11 


4.76 


1.15 


4.08 


1.35 




Nov3 


[1..7] 


4.75 


1.32 


4.65 


1.14 


3.87 


1.43 


Reputation 


Repl 


[1-7] 


3.62 


1.47 


4.11 


1.48 


2.68 


1.62 




Rep2 


[1..7] 


3.64 


1.56 


3.73 


1.44 


2.97 


1.72 




Rep3 


[1-7] 


4.20 


1 .57 


4.47 


1.37 


3.38 


1.76 


Intrinsic Motives 


Intl 


[1..7] 


6.00 


0.84 


5.76 


1.06 


5.18 


1.42 




Int2 


[1-7] 


6.38 


0.62 


6.05 


0.84 


5.89 


1.13 


Age 


Agel 




46.32 


15.26 


52.30 


15.56 


42.56 


14.09 


Gender 


Genl 


0/1 


0.78 


0.42 


0.97 


0.17 


0.94 


0.24 


Expertise 


Expl 


[1..7] 


5.36 


1.11 


5.87 


0.98 


5.56 


1.23 


Quantity of Contributions 


Qntl 


[1-7] 


5.33 


1.34 


4.77 


1.24 


4.48 


1.61 




Qnt2 


[1..7] 


5.42 


1.35 


4.63 


1.23 


4.82 


1.61 


Quality of Contributions 


Quail 


[0..1] 


0.67 


0.30 
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(computer expertise). Of the three models tested, clearly, the 
proposed (six factor) measurement model provided the best 
explanation for the observed variance and covariance among the 
set of self-reported indicator variables, and this result was 
consistent for the three studies at Stardust@home, CWOP, and 
BOINC. 

In addition, we employed Partial Least Squares (PLS) using the 
SmartPLS 2.0 software [73] to assess the reliability of our 
measures and the structural model. The PLS algorithm estimates 
path models using composite variables, sometimes called latent 
variables, from a number of indicator items, sometimes referred to 
as manifest variables. In this respect, the variance-based PLS path 
modeling is similar to covariance-based structural equation 
modeling (SEM) because both algorithms estimate complex 
relations between several latent variables simultaneously. Never- 
theless, a number of conceptual and formal differences make PLS 
path modeling especially suited for this study. Although both PLS 
and SEM may suffer when sample size is very small and with non- 
normally distributed data [74], the PLS algorithm performs better 
in these conditions and is more robust when assumptions of 
normality are violated [75,76]. This was an important consider- 
ation for choosing to use PLS in our study, given that some of the 
variables are not normally distributed. 

Using PLS, an index of internal consistency was computed for 
each multi-item scale. Composite reliability values of 0.89-0.96, 
0.89-0.98, and 0.89-0.95 were found at Stardust@home, CWOP, 
and BOINC respectively (see Tables 2-4). In addition, we 
analyzed the individual loadings items on their corresponding 
underlying factor, as well as by the Average Variance Extracted 
(AVE). All item loadings on their intended constructs were greater 
than 0.70 and substantially higher than the cross-loadings (see 
Appendix A). The AVE for each construct was 0.73-0.93, 0.73- 
0.93, and 0.73-0.91 at Stardust@home, CWOP, and BOINC 
respectively, substantially greater than the suggested threshold of 
0.50 [77]. See Tables 2-4 for details. 



We assessed discriminant validity by comparing the square root 
of the AVE (RAVE) of a particular construct (see Tables 2-4 on 
the diagonal, in bold) and the correlation between that construct 
and other latent constructs (presented by the off-diagonal position 
of the tables). We found that the constructs' RAVE ranges were 
0.85-0.97, 0.85-0.96, and 0.85-0.96 at Stardust@home, CWOP, 
and BOINC respectively, while correlations between constructs 
were generally below the recommended threshold of 0.5 (with two 
exceptions: at CWOP the correlations between Reputation and 
Quantity of Contribution was 0.58 and the correlations between 
norm-oriented and intrinsic motives was 0.51). In addition, RAVE 
for every construct is substantially higher than the correlation 
between that construct and all other constructs. Furthermore, the 
loadings of all items on their intended construct were higher than 
on other constructs (see Appendix). Having established reliable 
and construct valid measures, we tested the study hypotheses by 
assessing the extent to which the proposed (structural) model fit the 
observed pattern of variance and covariance among the study 
measures. 

Hypothesis Testing: Assessing the Fit of the Structural 
Model 

The specified paths in the structural model corresponded to a 
large extent to the hypothesized relationships. The significance of 
structural path estimates was computed using the bootstrapping 
re-sampling method with 500 re-samples. The structural model 
was evaluated on the basis of the statistical significance of 
structural paths and the R for each composite latent variable. 
Figure 1 show the results of the PLS analysis for the model 
explaining contributions' quantity at Stardust@home, CWOP, 
and BOINC . Figure 2 shows the results of the PLS analysis for the 
model explaining contributions' quality at Stardust@home, 
demonstrating a significant positive relations between contribution 
quality and both reputation and collective motives, and a negative 
relation with intrinsic motivation. In addition, a positive relation 
was found between reputation and intrinsic motives. 
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Table 2. Stardust@home: AVE, Composite Reliability, square-root of AVE (on diagonal; bold) and correlation between the latent 
constructs. 



Correlations 



Stardust @home 


AVE 


Comp Rel. 


Collective 


Norm 


Reputation 


Intrinsic 


Age 


Gender 


Expertise Quantity 


Quality 


Collective Motives 


0.88 


0.93 


0.94 
















Norm-Oriented Motives 


0.73 


0.89 


-0.09 


0.85 














Reputation 


0.76 


0.91 


-0.08 


0.13 


0.87 












Intrinsic Motives 


0.82 


0.90 


0.20 


0.17 


0.18 


0.91 










[Control] Age 


1.00 


1.00 


0.09 


-0.09 


-0.22 


-0.22 


1.00 








[Control] Gender 


1.00 


1.00 


0.26 


-0.03 


0.06 


-0.14 


-0.01 


1.00 






[Control] Expertise 


1.00 


1.00 


0.25 


0.14 


-0.02 


0.02 


-0.16 


0.01 


1.00 




Quantity of Contributions 


0.93 


0.96 


0.24 


0.30 


0.14 


0.45 


-0.33 


0.05 


0.17 0.97 




Quality of Contributions 


1.00 


1.00 


0.38 


-0.10 


0.07 


-0.02 


0.05 


0.07 


0.25 0.07 


1.00 
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Discussion 

This study focuses on online citizen science, in which TMSP is 
channeled to active participation in scientific research projects. 
The objective of this study was to investigate the relationships 
between various motivational factors and both the quantity and 
quality of contribution. Examining three different projects, 
representing a spectrum of task granularity levels, make the results 
more generalizable, illustrating the similarities and differences 
between different types of citizen science projects. The theoretical 
lens that served the basis for our study was Klandermans' model of 
participation in social movements [39]. 

Results from our study show that all four factors - collective 
motives, norm-oriented motives, reputation, and intrinsic motives 
- are important drivers of the contribution quantity. Together, the 
factors analyzed explained 36%, 43%, and 31% of the variance in 
the outcome variable (for Stardust@home, CWOP, and BOINC 
respectively). Moreover, these results were consistent across the 
three different citizen science settings (with one exception 
discussed below). 

While collective motives constitute an important component of 
Klandermans model, empirical findings from prior TMSP studies 
yielded mixed results in terms of the impact of these motives on 
participation. Our findings highlight the importance of these 
motives across different citizen science settings. 



Prior research suggests that norm-oriented motives play an 
important part in explaining participation in volunteering and 
social movements [39,48], and subjective norms are considered an 
important antecedent of the decision to adopt an information 
technology [49]. However, empirical studies both offline and 
online social participation that investigated the effects of social 
norms on participation have yielded inconclusive results ([46,51]). 
The findings of the present study support the notion that while 
perceived social pressure from important others could encourage 
volunteers to engage and contribute to citizen science projects, it is 
less likely to induce the kind of effort and commitment that are 
necessary for making high-quality contributions. 

Individuals often engage in social interaction and collective 
activity based on an expectation that it will lead in some way to 
personal rewards, such as enhanced reputation. In line with prior 
research in TMSP, reputation was shown to be positively related 
to volunteer participation in the CWOP and BOINC settings. 

The fourth antecedent, intrinsic motivation, was found to be 
positively related to contribution quantity in our three studies, 
reinforcing our understanding of voluntarism [41] and TMSP 
[11,15,16,42]. 

Another finding involves the comparison between the drivers of 
contribution quantity and quality. By using the same independent 
variables to study the differences between antecedents of 
contribution quantity and quality at Stardust@home, we show 



Table 3. CWOP: AVE, Composite Reliability, square-root of AVE (on diagonal; bold) and correlation between the latent constructs. 



Correlations 



CWOP 


AVE 


Comp Rel. 


Collective 


Norm 


Reputation Intrinsic 


Age 


Gender 


Expertise 


Quantity 


Collective Motives 


0.83 


0.91 


0.91 
















Norm-Oriented Motives 


0.73 


0.89 


0.27 


0.85 














Reputation 


0.79 


0.92 


0.19 


0.46 


0.89 












Intrinsic Motives 


0.81 


0.89 


0.36 


0.51 


0.40 


0.90 










[Control] Age 


1.00 


1.00 


-0.01 


-0.08 


-0.06 


-0.08 


1.00 








[Control] Gender 


1.00 


1.00 


-0.04 


-0.08 


0.01 


-0.06 


-0.01 


1.00 






[Control] Expertise 


1.00 


1.00 


0.16 


0.07 


-0.02 


0.08 


-0.11 


0.02 


1.00 




Quantity of Contributions 


0.93 


0.96 


0.26 


0.46 


0.58 


0.47 


-0.13 


0.00 


0.01 


0.96 
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Table 4. BOINC: AVE, Composite Reliability, square-root of AVE (on diagonal; bold) and correlation between the latent constructs. 



Correlations 



BOINC 


AVE 


Comp Rel. 


Collective 


Norm 


Reputation 


Intrinsic 


Age 


Gender 


Expertise 


Quantity 


Collective Motives 


0.91 


0.95 


0.96 
















Norm-Oriented Motives 


0.73 


0.89 


0.21 


0.85 














Reputation 


0.76 


0.90 


-0.05 


0.28 


0.87 












Intrinsic Motives 


0.79 


0.89 


0.40 


0.34 


0.23 


0.89 










[Control] Age 


1.00 


1.00 


0.03 


-0.10 


-0.06 


-0.07 


1.00 








[Control] Gender 


1.00 


1.00 


0.01 


0.04 


0.01 


0.05 


-0.01 


1.00 






[Control] Expertise 


1.00 


1.00 


0.04 


0.09 


-0.02 


-0.01 


-0.09 


-0.16 


1.00 




Quantity of Contributions 


0.90 


0.94 


0.20 


0.33 


0.39 


0.41 


-0.19 


-0.06 


0.12 


0.95 


doi:1 0.1 371 /joumal.pone.0090375.t004 



that while the effect of collective motives is similar for both 
quantity and quality, there are substantial differences between the 
effects of the other factors. Norm-oriented and intrinsic motives, 
which had a positive effect on contribution quantity, did not have 
a similar effect on quality; in fact, the effects of these factors on the 
quality of contributions were negative. Norm-oriented motives 
may play an important role in enforcing participation, but they 
may be insufficient for driving citizen scientists to exert the effort 
required for ensuring high-quality outcomes. In a similar manner, 
the results indicate that enjoyment may entice volunteers to 
engage with the project, but the additional investment necessary 
for executing the job well, deem intrinsic motives insufficient for 
ensuring contribution quality, in line with the findings from online 
communities [28]. In short, by and large, prior studies of TMSP 
have focused on the quantity of contribution as the primary 



outcome (for an exception see [14]). Our study demonstrates that 
more attention should be given to studying the factors driving 
contribution quality. 

Another interesting result of the comparison of quantity and 
quality involves the effect of reputation. While at the volunteer 
computing and weather monitoring settings reputation was found 
to be an important determinant of participation quantity, at 
Stardust@home reputation had only an insignificant effect on 
quantity while having a significant positive effect on contribution 
quality. A similar effect was observed at Flickr, where extrinsic 
motives had a positive significant effect on quality, but not on 
quantity [26]. We believe that the effects of reputation are 
determined to a large extent by what indicator of performance is 
made visible publicly. For example, at BOINC the credit of each 
user is calculated based solely on the quantity of; this information 




Figure 1. Results of PLS analysis - contribution quantity; Stardust@home/CWOP/BOINC. Please note that '*' p,0.05; p,0.01; and " 
p,0.001. 

doi:1 0.1 371 /journal.pone.0090375.g001 
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Figure 2. Results of PLS analysis - Stardust@home contribution quality. Please note that '*' p,0.05; '**' p : 0.01; and '***' p,0.001. 
doi:1 0.1 371 /journal.pone.0090375.g002 



is presented to the user and others in the community. At 
Stardust@home, on the other hand, a significant emphasis is 
placed on the quality of work. 

Our findings from all three settings - across different task 
granularity levels - reveal a 'crowding in' effect, whereby extrinsic 
motives (i.e. reputation) reinforce intrinsic motivation. The theory 
suggests that both 'crowding in' and 'crowding out' (the opposite 
effect, when external incentives are perceived to be controlling 
such that self determination is reduced and intrinsic motivation is 
undermined) are possible, and their manifestation depends on the 
specific organizational contexts [60-63,78]. Crowding effects have 
largely not been studied in the context of TMSP. Our findings 
show that in the three citizen science settings investigated - despite 
substantial differences the nature of volunteers' tasks and project 
administration mechanisms - reputation actually acts to enhance 
volunteers' intrinsic motives. Such an effect is typical of trusting 
environments that include reciprocal relationship between mem- 
bers and the organization [64,65], suggesting that citizen science 
projects have been successful at building a constructive environ- 
ment where the goals of citizen and professional scientists are 
aligned. 

Finally, the effects of the control variables on the quantity of 
contribution were highly consistent across the three settings: Age 
had a significant negative effect, indicating that younger people 
tend to contribute more frequently; while Gender and Computer 
Expertise had insignificant effects (except for the significant 
positive effect of participants' expertise at one setting: Stardust@ 
home). Overall, the controls made a relatively little contribution 
towards explaining the variance in the quantity of contributions: 
they added 6%, 1%, and 2% at Stardust, CWOP, and BOINC 
respectively. When studying the antecedents of contributions' 
quality at Stardust@home, Computer Expertise was the only 
control to have a (positive) significant effect, reflecting the relative 
complexity of the image analysis task. The three controls have 



contributed 3% towards explaining the variance in contribution 
quality. 

Conclusion 

The present study contributes to theory in two ways. First, it 
increases our understanding of what motivates citizen scientists. 
The second contribution goes beyond the unique citizen science 
context to the study of TMSP in general, and concerns the 
differences between contribution quantity and quality in their 
motivational underpinnings. Our findings from one citizen science 
projects provide preliminary evidence that factors that enhance 
participation frequency may not necessarily lead to enhanced 
contribution quality, and in some case actually detract from 
quality, calling for a more careful investigation of the effects of 
motivation. 

Information systems research has highlighted the need for 
contextualizing theory for a specific domain. We argue that given 
the variety of settings, modalities and content domains in which 
TMSP is practiced, a context-specific approach is required for 
TMSP theorizing. Questions of why people participate in social 
movements can be approached from both macro-level and micro- 
level perspectives [46] . Thus, an approach that incorporates both 
micro-level (e.g. motivational) and macro-level (e.g. organizational) 
factors is needed. 

The findings also have important implications for the design 
and management of online citizen science projects. First, the 
positive effects of collective motives across projects and outcome 
variables suggest that citizen science projects should strive to 
increase volunteers' commitment to the project and its goals. This 
could be done by communicating the project's mission and 
achievement to the volunteers. Second, while all motivational 
factors had a positive effect on the level of participation, the 
salience of factors differs between projects. Special attention 
should be therefore given to the unique characteristics of each 
project setting, and to the effects of these features on motivation. A 
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third implication concerns the factors that determine the quality of 
contribution. For example, the fact that intrinsic motivation was 
not found to enhance quality stresses the need to develop more 
enjoyable, game-like, participation mechanisms, such as the one 
used in Foldit [6]. Similarly, mechanisms such as social network 
features should be put in place to create and emphasize social 
influences, linking them to the quality of one's contributions, so 
that norm-oriented motives would be positively linked to 
contribution quality. Finally, motivations could potentially be 
enhanced by creating a dynamic organization that would allow 
volunteers to gradually become more involved, and assume 
responsibility and authority. Many TMSP efforts, such as OSS 
projects and Wikipedia, have long realized this, and enable 
volunteers to progress in the tasks they perform and the 
responsibilities they assume [79]. This mechanism is currendy 
absent in online citizen science projects, where volunteers' tasks 
are usually restricted in scope, and the governance and decision- 
making is left in the hands of the professional scientists. Adopting a 
more nuanced governance structure in which high-performing 
volunteers are more empowered, would not only enhance citizen 
scientists' motivations, but may also reduce the load on 
professional scientists who could delegate some tasks, such as 
quality control, to volunteers with established track record. Such a 
change represents a major shift in thinking and theorizing about 
the relationships between science, scientists and society. However, 
as online citizen science develops and matures, and demand for 
volunteers' resource increases, such a trend toward greater 
volunteer empowerment may be inevitable. 

Conclusions drawn from this study should be considered in light 
of several limitations. First, although using a convenience sample 
for testing basic psychological mechanisms is often used in 
information systems research [80], it does limit to some extent 
the generalizability of the study's findings. Nonetheless, there are 
no plausible reasons to suggest that volunteers in our sample 
should differ substantially from the overall population of citizen 
scientists. Second, there may be some unique features to the three 
citizen science projects we studied, as there are such features to 
any online community in general. Future studies of citizen science 
projects in different fields, with different goals, could help verify 
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